A  TESTBED  FOR  NUCLEAR  EXPLOSION  MONITORING  RESEARCH  AND  DEVELOPMENT 


Robert  L.  Woodward,  David  H.  Salzberg,  Robert  G.  North 
Center  for  Monitoring  Research 
Science  Applications  International  Corporation 

Sponsored  by  The  Defense  Threat  Reduction  Agency 
Arms  Control  Technology  Division 
Nuclear  Treaties  Branch 

Contract  No.  DTRA01-99-C-0025 


ABSTRACT 


The  Research  and  Development  Testbed  (R&D  Testbed),  developed  within  the  Defense  Threat  Reduction  Agency’s 
(DTRA)  Center  for  Monitoring  Research  (CMR),  is  designed  to  support  US  national  and  international  nuclear  test 
monitoring  and  verification  needs  by  providing  several  key  functions  to  the  R&D  community.  First,  the  R&D 
Testbed  supports  the  Department  of  Defense  (DoD)  sponsored  research  and  development  community  by  providing 
an  interface  to  the  wide  range  of  resources  available  at  CMR.  For  example,  the  R&D  Testbed  provides  access  to 
CMR  resources  such  as:  near -real  time  waveform  data  from  International  Monitoring  System  (IMS)  and  GSETT3 
stations,  multi-terabyte  seismic,  hydroacoustic  and  infrasonic  waveform  archives,  specialty  ground-truth  calibration 
databases,  research  archives,  high  resolution  gamma-ray  spectroscopy  and  beta-gamma  coincidence  spectroscopy 
radionuclide  databases,  and  past  and  present  products  of  both  the  Prototype  International  Data  Center  and  the 
International  Data  Centre  (IDC). 

A  second  R&D  Testbed  function  is  to  provide  a  centralized  repository  and  exchange  mechanism  for  research  results 
generated  by  DoD  sponsored  researchers,  facilitating  the  exchange  of  results  among  the  R&D  community.  Uniform 
acceptance  criteria  and  validation  testing  are  applied  to  all  results  delivered  to  the  R&D  Testbed,  thus  ensuring  easy 
and  reliable  use  of  these  results  by  others.  Further,  centralized  storage  ensures  that  valuable  research  results 
accumulate  through  time— preventing  their  loss  and  avoiding  their  duplication.  Deliveries  to  the  R&D  Testbed  can 
take  a  variety  of  forms  such  as:  technical  reports,  waveform  data,  event  catalogues,  parametric  information, 
algorithms,  and  software.  The  mechanics  of  delivering  results  to  the  R&D  Testbed  is  the  subject  of  another 
symposium  presentation. 

Finally,  R&D  Testbed  facilities  provide  researchers  with  the  tools  to  evaluate  R&D  results  to  assess  their  impact  on 
monitoring  capability  by  providing  test  environments  at  whatever  scale  is  appropriate  to  the  evaluation.  Testing  can 
range  from  the  full  data  load  of  the  IMS  stations  and  the  full  processing  environment  of  the  IDC,  to  highly  specific 
tests  confined  to  special  data  sets.  This  wide  range  of  testing  environments  provides  unique  testing  opportunities  for 
the  R&D  community.  In  particular,  testing  at  operational-like  scales  allows  issues  to  be  identified  which  can  go 
unnoticed  in  more  limited-scale  testing.  The  evaluations  of  R&D  results  will  be  passed  to  DTRA  so  that  the  most 
promising  R&D  products  can  be  identified  for  appropriate  action,  such  as  further  development  or  integration  into 
larger  systems. 

The  R&D  Testbed  is  a  collaboration  between  DTRA,  the  R&D  community,  and  CMR.  A  wide  range  of  CMR  staff, 
including  scientific,  software  development,  testing,  and  infrastructure  support  teams  are  available  to  support  all 
phases  of  R&D  Testbed  activity.  Although  for  many  DTRA  contractors  the  delivery  of  results  to  the  R&D  Testbed 
is  mandatory,  the  R&D  Testbed  staff  will  work  with  contributors  on  advance  planning,  scheduling,  assisting  with 
test  plans  and  other  activities  to  make  the  delivery  and  acceptance  process  as  smooth  as  possible.  Such  coordination 
among  all  participants  is  crucial,  as  the  R&D  Testbed  will  be  receiving  deliveries  of  various  types  from  nearly 
ninety  different  research  contracts  over  the  next  three  years. 
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MISSION 


The  mission  of  the  DTRA  CMR  R&D  Testbed  is  to  provide  a  research  and  development  environment  with  test  and 
evaluation  facilities  at  appropriate  scales  to  enable  advances,  measure  progress,  and  focus  resources  on  the  important 
issues  in  monitoring  and  verification.  To  achieve  this  mission,  a  wide  range  of  resources  and  experience  at  CMR  are 
available  to  the  nuclear  monitoring  research  community.  In  the  remainder  of  this  paper  we  highlight  some  of  these 
resources  and  how  they  may  be  utilized  to  support  both  research  and  development  efforts,  as  well  as  the  test, 
evaluation,  and  transition  of  subsequent  research  results. 

BACKGROUND 


The  R&D  Testbed  at  the  Defense  Threat  Reduction  Agency’s  Center  for  Monitoring  Research  provides  a  valuable 
resource  to  the  nuclear  monitoring  R&D  community.  The  CMR  facility  has  a  long  record  of  research  and 
development  activities,  supporting  both  in-house  and  external  research.  Further,  the  processes  and  procedures 
utilized  by  CMR  are  based  on  a  history  of  moving  solutions  from  basic  R&D  to  integrated  systems.  For  example, 
over  the  past  several  years  CMR  has  been  developing,  testing,  and  transitioning  systems  to  both  US  government  and 
multi-lateral  monitoring  organizations  (e.g.  the  US  National  Data  Center  located  in  Florida  at  Patrick  Air  Force  Base 


Figure  1.  Illustration  of  the  process  for  moving  from  basic  and  exploratory  R&D  work,  on  the  left  of  the 
figure,  to  final  operational  systems,  on  the  right  side  of  the  figure.  As  one  moves  from  basic  R&D  testing  to  the 
full-scale  test  environment,  the  formalism  of  the  procedures  increases,  proportional  to  the  scale  of  the  testing. 
In  addition,  this  figure  illustrates  how  the  PIDC  7.0  software  release  feeds  into  the  NDC  operations 
environment,  and  Release  3  software  (based  on  PIDC  7.0)  feeds  into  IDC  operations  environment. 


(NDC),  and  the  International  Data  Centre  located  in  Vienna,  Austria  (IDC).  As  illustrated  in  Figure  1,  these  efforts 
have  resulted  in  refined  procedures  for  all  stages  of  the  process  required  to  move  basic  R&D  results  from  the 
research  stage  through  to  operational  systems.  Extensive  testing  is  conducted  at  all  scales  as  a  routine  part  of  the 
development  and  transition  process.  These  tests  range  from  stand-alone  and  unit  tests  of  individual  software 
components  and  parametric  information,  to  integration  testing  on  an  isolated  testbed,  up  to  operational  test  and 
evaluation  in  a  full-scale  environment. 


DATA  AND  ANALYSIS  RESOURCES 


The  R&D  Testbed  provides  access  to  a  variety  of  data  and  products,  many  of  which  are  produced  by  the  Prototype 
International  Data  Center  (PIDC)  at  CMR.  A  brief  listing  of  some  of  the  more  important  data  resources  includes: 

•  Waveform  data  (seismic,  infrasound,  hydroacoustic) 

•  Radionuclide  data 

•  Event  bulletins  (from  both  the  PIDC  and  IDC) 

•  Special-purpose  event  and  waveform  databases 

•  Web  pages 


Most  data  and  results  can  be  accessed  remotely  using  AutoDRM  or  web-based  interfaces.  See  the  R&D  Testbed  web 
site  (|http://www.cmr. gov/rdtb)  for  more  details  and  additional  links.  In  addition,  open  international  facilities  are 
available  for  visiting  scientists  who  wish  to  directly  access  research,  operations,  and  development  expertise,  as  well 
as  interact  closely  with  the  CMR  data  archives  and  databases.  Computer  and  database  accounts  are  also  available  for 
direct  access  to  R&D  Testbed  resources. 


CMR  currently  receives  over  three  gigabytes  of  data  from  the  IMS  and  other  networks  each  day  and  has  over  five 
years  of  archived  data.  Continuous  time  series  data  arrive  in  near-real-time  from  primary  seismic  stations  and  arrays, 
as  well  as  from  hydroacoustic  and  infrasound  stations.  Waveform  segments  are  requested  from  auxiliary  seismic 
stations,  and  radionuclide  stations  send  in  observations  as  discrete  messages.  Figure  2  illustrates  the  quantity  of 
seismic,  hydroacoustic  and  infrasound  data  received  since  1995.  The  R&D  Testbed  provides  complete  access  to  the 
data  archives,  as  well  as  current  data  arriving  via  the  International  Monitoring  System  (IMS),  though  the  latter  are 
subject  to  some  restriction. 


Gigabytes  of  data  per  year 


year 

Figure  2.  Gigabytes  of  data  in  the  CMR  data  archive.  The  majority  of  the  data,  in  terms  of  volume,  are 
comprised  of  continuous  time  series  data  from  seismic,  hyrdoacoustic,  and  infrasound  sensors. 


Processing  results  from  the  PIDC  for  all  of  the  data  collected  since  1995  are  available  on-line  and  by  remote  access. 
These  results  include  preliminary  automated  event  bulletins  as  well  as  the  analyst  Reviewed  Event  Bulletin  (REB). 
As  the  IDC  has  assumed  production  of  daily  REBs  since  February  20,  2000,  the  PIDC  has  scaled  back  to  an  average 
of  two  REBs  produced  each  week.  The  R&D  Testbed  provides  the  research  community  with  access  to  both  PIDC 
and  IDC  REBs  for  every  day,  from  the  present  back  to  January  1,  1995,  comprising  over  100,000  events. 

A  variety  of  databases  have  been  assembled  for  research  purposes,  and  these  are  best  accessed  via  the  web  (see  the 
R&D  Testbed  web  site,  www.cmr.gov/rdtb,  for  links  and  instructions).  For  example,  the  Nuclear  Explosion 
Database  contains  a  large  set  of  seismic  recordings  of  nuclear  tests.  The  Ground  Truth  Database  contains  an 
extensive  catalog  of  events  for  which  seismic  ground  truth  information,  to  various  levels  of  certainty,  are  known. 
Although  the  seismic  R&D  databases  are  most  numerous  at  CMR,  it  is  important  to  note  that  databases  exist  for  all 
four  monitoring  technologies:  There  are  hydroacoustic  and  infrasound  research  databases,  as  well  as  high  resolution 


gamma-ray  spectroscopy  and  beta-gamma  coincidence  spectroscopy  radionuclide  databases.  Reports  are  available  to 
accompany  most  of  the  special  databases  and  augment  the  detail  provided  on  the  web  site. 

DELIVERIES  TO  THE  R&D  TESTBED 


The  principal  investigators  of  DTRA-sponsored  nuclear  monitoring  research  have  been  instructed  to  deliver  the 
results  of  their  research  to  the  R&D  Testbed.  Deliveries  take  a  number  of  forms,  such  as  reports,  raw  data  (e.g. 
seismic  waveforms),  parametric  information  fe.g.  velocity  models,  station  correction  tables),  and  software.  The 
R&D  TB  verifies  that  all  deliveries  meet  appropriate  acceptance  criteria  and  pass  validation  tests.  Such  screening  of 
contributions  ensures  that  deliveries  have  been  made  successfully  and  increases  the  probability  that  these  results 
may  be  successfully  used  by  others.  In  addition,  the  R&D  Testbed  acts  as  an  archive  of  research  results,  to  prevent 
their  loss  and  facilitate  their  distribution  within  the  R&D  community.  Finally,  the  accumulation  of  results  through 
time,  and  across  projects  and  disciplines,  provides  the  users  of  the  R&D  Testbed  with  a  unique  research  tool. 

Various  procedures  have  been  established  for  making  deliveries  to  the  R&D  Testbed,  as  well  as  using  the  testbed  for 
other  purposes,  such  as  testing  and  evaluating  research  results  (Salzberg,  et  al,  this  issue).  Test  and  evaluation 
procedures  are  scaled  to  the  level  necessary  to  achieve  meaningful  results.  For  example,  the  most  stringent  criteria 
are  applied  to  the  use  of  the  full-scale  testing  environment,  while  more  relaxed  criteria  are  appropriate  for  limited- 
scale  tests  or  the  test  and  evaluation  of  stand-alone  applications.  An  oversight  committee  prioritizes  R&D  Testbed 
activities,  as  well  as  reviewing  schedules,  plans  and  results. 

TEST  ENVIRONMENTS 


Multiple  test  environments  are  available  as  part  of  the  R&D  Testbed.  The  smallest-scale  test  environments  are 
readily  reconfigured  and  adapted  to  the  specific  needs  of  a  given  test.  At  the  largest  scale  is  the  capability  for  full- 
scale  testing  under  heavy,  real-time,  data-flow  and  processing  loads.  The  key  benefit  of  the  R&D  Testbed’s  overall 
test  capability  is  illustrated  in  Figure  3.  Tests  can  be  flexible  enough  to  provide  direct  R&D  interaction  and 
experimentation,  yet  can  be  structured  enough  to  simulate  a  full-scale,  mission-critical  operational  environment. 
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Figure  3.  Illustration  of  the  range  of  test  environments  available  within  the  R&D  Testbed. 


Full-scale  testing  capability  is  provided  by  the  Prototype  Data  Center  (PDC),  which  comprises  a  superset  of  the 
functionality  of  the  PIDC.  Tests  using  the  PDC  are  subjected  to  the  full  data  load  and  timeliness  pressures  of  a 
system  receiving  and  processing  continuous  data  running  on  a  high  availability  basis.  Strict  configuration  control  is 
applied  to  the  full-scale  system,  with  changes  subject  to  the  approval  of  the  Configuration  Control  Board.  Such 
control  ensures  that  changes  in  behavior  of  the  system,  or  system  output,  are  well  understood,  predictable,  and 
repeatable. 

A  second  full-scale,  but  more  configurable,  system  is  provided  by  the  integration  testbed.  The  integration  testbed 
system  completely  replicates  the  PDC  full-scale  system  (with  the  exception  of  the  mass  storage  system),  and 
provides  an  environment  which  can  be  configured  for  specific  tests.  Currently,  the  integration  testbed  is  used  for 
integrating  and  testing  the  full  CTBT  Monitoring  Software  System  releases  for  the  PIDC,  as  well  as  other  tests 
which  are  incompatible  with  the  more  formal  PDC  environment  or  where  special  configurations  are  required. 


Finally,  the  R&D  Testbed  provides  a  broad  range  of  environments  for  unit  tests  and  stand-alone  tests.  These  tests 
take  advantage  of  the  available  resources  and  processing  results  and  range  in  scope  from  single  desktops  to  complete 
systems. 

FACILITIES 


A  comprehensive  infrastructure  environment  exists  at  CMR  to  support  the  R&D  Testbed  user  community  and  the 
R&D  Testbed  itself.  A  substantial  Commercial  Off  the  Shelf  (COTS)  software  and  hardware  environment  is 
maintained  and  documented  on  the  R&D  Testbed  web  site,  and  is  highlighted  here: 

•  Over  60  servers 

•  Over  90  workstations 

•  Mass  storage  systems  (DLT,  CD-ROM) 

•  Multiple  Oracle  instances 

•  Multiple  web  sites  (open  and  secure) 

•  Multiple  networks 

•  Firewall  security 

A  number  of  these  resources  can  be  utilized  directly  by  the  R&D  community  (e.g.  accounts  are  available  for  direct 
use  of  the  Oracle  database),  while  others  are  used  to  provide  community  support  (e.g.  the  mass  storage  system  is 
used  to  archive  and  serve  data). 

R&D  Testbed  contributors  can  conduct  software  demonstrations,  as  well  as  train  others  in  the  use  of  their  software, 
in  the  training  and  briefing  facility.  There  are  multiple  large-screen  computer  projections,  from  both  Sun  UNIX  and 
PC  hardware  platforms,  with  access  to  the  complete  CMR  COTS  software  environment. 

SUMMARY 


The  R&D  Testbed  makes  use  of  all  the  facilities,  resources,  and  expertise  of  CMR  for  the  purpose  of  improving 
nuclear  monitoring  and  verification  capabilities.  This  role  can  be  summarized  in  terms  of  the  following  broadly 
stated  goals: 

•  The  R&D  Testbed  supports  the  R&D  community,  from  the  inception  of  research  through  to  testing  and 
archiving  of  results  on  the  R&D  testbed  itself.  Such  support  can  take  the  form  of  providing  access  to  current 
IMS  data,  production  results,  IDC  results,  data  archives,  and  more. 

•  The  R&D  Testbed  archives  research  results  to  ensure  important  work  is  not  lost  and  can  be  shared  readily 
amongst  the  R&D  community. 

•  The  R&D  Testbed  provides  a  mechanism  for  the  test,  evaluation,  and  integration  of  R&D  results.  Testing  can  be 
conducted  at  various  scales,  ensuring  proper  test  environments.  Such  test  and  evaluation  provides  the  US 
government  with  quantitative  measures  of  current  monitoring  capability,  identifying  obstacles  and  establishing 
expectations  for  achieving  improved  capability. 


